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Abstract 

Necessary conditions of optimality in the form of the Pontryagin Maximum Principle 
are derived for the Bolza-type discounted problem with free right end. The optimality is 
understood in the sense of the uniformly overtaking optimality. Such process is assumed 
to exist, and the corresponding payoff of the optimal process (expressed in the form of 
improper integral) is assumed to converge in the Riemann sense. No other assumptions 
on the asymptotic behaviour of trajectories or adjoint variables are required. 

In this paper, we prove that there exists a corresponding limiting solution of the 
Pontryagin Maximum Principle that satisfies the Michel transversality condition; in par¬ 
ticular, the stationarity condition of the maximized Hamiltonian and the fact that the 
maximized Hamiltonian vanishes at infinity are proved. The connection of this condition 
with the limiting subdifferentials of payoff function along the optimal process at infinity 
is showed. The case of payoff without discount multiplier is also considered. 

Keywords: Infinite horizon problem, transversality condition for infinity, Pontrya¬ 
gin maximum principle, Michel condition, Limiting subdifferential, Uniformly overtaking 
optimal control, Shadow prices. 
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Introduction 

The main means of construction of necessary conditions of optimality for control problems is 
the Pontryagin Maximum Principle |22j . In case of infinite horizon, the maximum principle is 
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generally incomplete (see mi): its relations offer no boundary condition at infinity. In absence 
of such transversality conditions, the PMP provides too many purportedly optimal solutions. 

Presently, many varieties of such conditions are constructed; a reference to all of them is not 
our intention. Nevertheless, let us note n El El m ES EH EB EZl ££] . One of such transversality 
conditions was proposed by Michel [19]. If the dynamics of the equation is autonomous and 
the payoff is of the form e~ rt fo(x, u ) dt , the condition may be rendered as 

/ OO 

e- rt fo(x*(t),u*(t))dt VT> 0, 

where ("0*, A*, x*, u*) satisfies the Pontryagin maximum principle. Like the other transver¬ 
sality conditions for infinite horizon, it only becomes a necessary condition under additional 
assumptions, see [21 Sect. 6]. There are many papers that prove the necessity of such conditions 
under various assumptions. In [31], the necessity was proved without assuming the dynamics 
to be smooth; in [15], it was studied in the calculus of variations setting; see [2D] for infinite 
horizon control problem with state constraints; in [25] it was proved for the general statement, 
including the problems with fixed right end; in [2], under sufficiently weak assumptions on 
the summability, the connection of this condition with the Aseev-Kryazhimskii formula was 
studied. The assumptions used in this paper could not be embedded into assumptions of the 
above-mentioned papers; in particular, in contrast with here, as well as in [TP] , 

the case of A* = 0 is not generally excluded. 

Note that the Michel condition, if convenient, is only one-dimensional and, therefore, this 
condition, together with the core conditions of the maximum principle, can determine a unique 
solution candidate only for the problems with one state variable. In view of that, it is important 
to know not only when this condition is necessary but also when it is consistent with other 
transversality conditions. For a similar analysis of the Aseev-Kryazhimskii formula, refer to 
[2]. Here, the Michel condition is used along with some limiting solution of the Pontryagin 
maximum principle (see HZ!); the limiting solution may be considered without assumptions on 
the asymptotic behaviour of trajectories or adjoint variables. The idea of the limiting solution 
can be traced to paper [23]; see its connection with the Aseev-Kryazhimskii formula in [T6j . 
The general case of Bolza-type infinite horizon problem with free right end was studied in 
H7]. In this paper, we prove the existence of a limiting solution of PMP that satisfies the 
Michel condition for uniformly overtaking optimal control; the arising transversality conditions 
are expressed in the form of limiting gradients of payoff function at infinity. The proof itself 
combines the ideas from [19] with the proof of the Pontryagin Maximum Principle from mi- 
The paper is structured as follows. First we describe the problem statement, impose the 
general conditions and propositions; at the same section, we provide the required definitions 
from the smooth analysis. In Section (2] in addition to the PMP relations and definition of a 
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limiting solution to the Pontryagin maximum principle, we specify the computation of limiting 
gradients of the payoff function at infinity. In the next section, we formulate the main result 
(Theorem [3]) and a number of its simple corollaries. The last two items contain, respectively, 
the preliminary lemmas and the proof of Theorem [3J 


1 Problem statement and definitions 


We consider the time interval T = M> 0 . The phase space of the control system is the finite¬ 
dimensional Euclidean space X = R m . 

Consider the following optimal control problem 


Minimize 1(b) + 


e rt fo(x, u)dt 

(la) 

■,u), u £ U, 

(lb) 

x(o) £ e. 

(lc) 


Here, the function f 0 is scalar; x is the state variable taking values in X; and u is the control 
parameter. 

Suppose that U is a Borel subset of a finite-dimensional Euclidean space. As for the class 
of admissible controls, we consider the set of measurable functions u(-) bounded for any time 
compact such that u(t) £ U holds for a.a. t £ T. Denote the set of admissible controls by IX. 

We assume the following conditions hold: 

• C is a closed subset of X; 

• l is taken to be locally Lipshitz continuous on x ; 

• / is Borel measurable in u and continuously differentiable in x ; 

• for each admissible control u, the map (t,x) H > f(x,u(t )) satisfies the sublinear growth 
condition (see, for example, [28], 1.4.4]); 

• /o is measurable in u , continuously differentiable in x , and lower semicontinuous in u ; 

• Tv-, are measurable in u and locally Lipshitz continuous on x. 

For each admissible control u , and position b 6 X, we can consider a solution of (116 li for 
x(0) = b. The solution is unique and it can be extended to the whole T. Let us denote it 
by x(b,u; •)■ 

The pair (x,u) will be called an admissible control process if u £ if, x(0) £ C, x(-) = 
ar(a?(0), tt; •)• 
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Definition 1 If an admissible process (x*,u*) satisfies 


lim sup 

T—> oo 


l(x*( 0)) + / e rt f 0 (x*(t),u*(t)) dt- 


inf 

(b,u)£Cxll 


+ 


e rt f 0 (x(b,u;t),u(t))dt 


< 0 , 


call it a uniformly overtaking optimal process for (I la I) - (I left 


Hereinafter assume there exists an optimal uniformly overtaking process (x*,u*) . Set 5* = 
x*(0). We are not going to impose any conditions that guarantee the existence of such a solution; 
for various existence theorems, refer to, for example, Mura. 

Let the improper integral 

e~ rt f 0 (x*(t),u*(t )) dt 

converge in the Riemann sense, i.e., 

</** = lim [ e~ rt f 0 (x*(t), u*(t)) dt G R. (2) 

t^oo J 0 

Let us now define scalar functions J°, J° by the following rule: for all b G X, T, s > 0, 

J°(b, s; T) = f e~ r{t+s) f 0 (x(b,u*]t),u(t)) dt. 

J o 

J°(6; T) = J°(b, 0; T) = [ e~ rt f 0 (x(b, u*;t), u(t)) dt. 

Jo 

To continue, we need to define subgradients of these payoffs at infinity. To this end, let us 
introduce the necessary notions of convex analysis [I2].[29l Sect.4], 

Consider a finite-dimensional Euclidian space E , and a lower semicontinuous function g : 
E — ^MU{+oo}. A vector f 6 E is said to be a proximal subgradient of g at b e E if there 
exist a neighborhood of b and a number a > 0 such that g(£) > g[b) + (,{£,— b) — cr||£ — ^l| 2 
for all £ 6 fh The set of proximal subgradients at b is denoted dpg(b ), and is referred to as the 
proximal sub differential. This set is nonempty for all b in a dense subset of {b\g(b) < +oo}. 
Following [29, Theorem 4.6.2(a)], denote the limiting sub differential of g at b by d^gfb) ; it 
consists of all ( in E such that 

3 sequences of y n 6X,(„6 d P g(y n ),y n b, ( n ^ (■ 

Following [291 Theorem 4.6.2(b)], denote the singular limiting (asymptotic limiting) subdiffer¬ 
ential of g at b by d° L g{b) ; it consists of all ( in E* such that 

3 sequences of y n G E, X n e T, ( n G d P g(y n ),y n b, A n j. 0, \ n ( n -> (. 
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If g is Lipshitz continuous near b, then d p g(b) is nonempty, moreover cod p g(b ) = 
d C iarkeg{b),d° L g(b ) = {0} (see [29, Sect. 4]). 

Following the same idea, define the subgradients of g at infinity, or, more accurately, along 
on arbitrary unboundedly increasing sequence of positive r. Fix a sequence r. 

Denote T = {r n \ n G N}. For a differentiable function g : E X T —> X, similarly to the 
definitions of limiting subdifferential and singular limiting sub differential, let us introduce the 
generalized subdifferential of g at the infinite point (6, oo T ), or rather at b with infinity along 
r , by the following rule: 

d\g{b , oo T ) = {C | 3 sequences of y n G E, t n GT,(„G d P g(y n , t n ), 

Dn b,t n OO, C n C}- 

Since in the general case it may be empty, let us also introduce a singular subdifferential in the 
following way: 

d° L g(b, oo T ) = {C| 3 sequences of y n G E,t n e 7, X n G T,(„ G d P g(y n , t n ), 

Un b,t n — >■ OO, X n —> 0, X n ( n —> C}- 

Note that the mentioned definitions can be rewritten otherwise. First of all, in the last two 
definitions, d P g(b,t n ) can be replaced with di,g(b,t n ) because every element from d p g(b,t n ) 
can be approximated with arbitrary precision by an element from di,g{y,t n ) for some y that 
is arbitrarily close to b. Moreover, in the definition of d\g , one can replace ( n —> C with 
AnCn C under the condition X n —> 1. Thus we obtain the equivalent form: 

d p g(b, oo T ) = {C| 3 sequences of y n G E,t n G T, X n G T, C n e d\g(y n ,t n ), 

Vn b,t n —)■ oo, A n —> A, A n Cn ~t C} VA G {0,1}. 

Remember that the limiting normal cone Nf(b) of C at b is the limiting sub differential 
d p Se(b ) of the indicator function Sq of the set C (see, for example, [TSj, Proposition 1.18]). 

2 Limiting solution of the Pontryagin Maximum Princi¬ 
ple 

Let us now proceed to the relations of the Pontryagin Maximum Principle. 

Let the Hamilton-Pontryagin function H : Xxf/xXxTxT i—)■ R be given by 
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(3a) 

(3b) 

(3c) 


H(x,u,if, X,t) = iff(x,u ) — Ae rt fo(x,u). Let us introduce the relations: 

x(t) = f(x(t),u(t)); 
dH 

~if(t) = -Q^{x(t),u(t),if(t),X,t)-, 

sup H(x(t),u',if(t), X,t) = H(x(t), u(t), if(t), A, t ); 
u'eu(t) 

for norming, it would also be convenient to use one of the following conditions: 


(3d) 

(3e) 


ll^(0)|| + A — 1; 

A G {0,1}. 


It is easily seen that, for each u G U for each initial condition, system (1 3a p - (1 36 [) has a local 
solution, and each solution of these relations can be extended to the whole T. 


Remark 1 Since the right-hand side of (1 46 1) - (1 4rf |) is homogeneous by (if, A), a nontrivial 
solution of (1 3 a II - (1 3c p with (1 3d |) exists iff there exists a nontrivial solution of (1 3a p 
with (1 3e p . 


Although the PMP holds for a rather general infinite-horizon control problem [13], its system 
of relations (1 3a J) - (13rf|) is generally incomplete and requires an additional boundary condition. 
Many such conditions, which hold under various supplementary assumptions (imposed, first of 
all, on the asymptotic behavior of the adjoint variable) were offered (see the reviews in [21 [SI]). 
In the general case, the result below does not require such assumptions [16]: 


Theorem 1 Let the process (x*, u*) be a uniformly overtaking process for problem (I la I) - (I Tc I) 
with singleton C . Let r be an unbounded increasing sequence of positive numbers. 

Then, for (x*,u*) there exists a t- limiting solution (if*, A*) of system (1 3a p - (1 3c I) satis¬ 
fying (CM} . 


Definition 2 A nontrivial solution (A *,if*) of (1 3a II - (1 3c I) associated with (x*,u*) is called 
t -limiting (or just limiting) if, for some subsequence r' C r, (x*,if*,\*) is a pointwise limit 
of solutions (x n ,if n , X n ) of the boundary value problems 


X n (t) 

= f(x n (t),U*(t))l 

(4a) 

-'if n(t ) 

= -Q^-{x n (t),u*(t),if n (t),X n ,f); 

(4b) 

Art (t) 

= 0; 

(4c) 

V’rt(r') 

= 0 

(4d) 


on the interval [0, r^]. 
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Remark 2 Without loss of generality we can say that X n + ||0 Tl (O)|| = A* + ||'0*(O)|| = 1. Or, 
if A* > 0, then we can say that X n = X* = 1. 


This definition of r-limiting solution (if*, A*) of system (1 3a [) - f l 3d I) has several equivalent 
formulations. 

First of all, let us use the fact that system (1 46 1) - (1 4c [) is linear. Denote by L the linear 
space of all real m x m matrices; here m = dim X. For each ( 6 X, there exists a solution 
A(£;t) G C(T, L) of the Cauchy problem 

= „*;(), ^tt;0) = 1 L . 

Define the vector-valued function / of time by the following rule: for every T G T, 


/({; T) = £ u"; t).u’(t)) A((; t) dt. 

Now, a solution (x n ,if n ,X n ) of system (1 4a I) - fl 4c I) satisfies the Cauchy formula: 


if(t) — (0(0) + A/(x(0); t))A 1 (x(0);t) Vt G T. (5) 

Note that thanks to if n (T n ) = 0 , we have if n (t) = A n (/(a; r) ,(0); t)—l(x n (0); r n ))A _1 (a; n (0); t); 
in particular, 

-0^(0) = —A n /(z n (0); r n ). (6) 

Passing to the limit and using the expression for / specified before, one can obtain the 
formulas for 0*(O). In particular, if there exists a finite limit of I(b]t) as b —> b*,t —^ oo, 
we see that the limiting co-state arc is unique up to a positive multiplication and the Aseev- 
Kryazhimskii formula holds: 

—ip*(0) — j e~ rt ^-(x*(t),u*(t)) A(b*;t) dt, X* = 1 . 

Assumptions under which this expression is a necessary condition of optimality that are rela¬ 
tively easy to check may be found in [2, fib] . This formula may not point towards a solution 
of the PMP even if the integral converges in the Lebesgue sense, see m For details on the 
other (the more general formulas), see |16j . 

To make an all-encompassing formulas of limiting co-state arc one can use the terms of 
limiting sub differentials of the payoff function J° at infinity: in D3 Theorem 3.1] it was 
proved that 


Theorem 2 A solution (if*, A*) of (1 3a I) - (1 3c p . (1 3e p associated with ( x*,u*) is t- limiting 
iff a solution (if*, A*) of (1 3a p - (1 3c I ) , (1 3e I ) associated with ( x*,u*) is nontrivial, and satisfies 


if*(0) E d£(~J°)(x*(0y,u*,oo T ). 
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In following, we show that the condition flTTj) implies the similar formula for the Hamilto¬ 
nian, or rather the pair (if, —H). 

For every d > 0 , define a control u G II by the rule u*~^(t) = u*(t + d). Now, for every 
b G X, there exists (GX such that x(f,u*]d) = b. Then, x(f, w*; d +1) = x(b,u*~' a ]i) for all 
t > 0. We can now provide a dehnition valid for all s G R, T > d : 

A*>, s; T) = J°(e, s; T) - J°(£, 5; tf) J* (x*; T) = 0; T). 


Note that, for all T G T, 
dJ° 


dJ° 


(b, s; T) = e~ rs I(b ; T), -—(6, s; T) = -r J u (6, s; T). 


Now, for every solution of system (1 4a p - (1 4c I) , the following identities hold: 
d I® 

(x n (d),s;T) = -rJ^(x n (d),s-,T); 

d 


(7) 


ds 
dJ* 


1" dx(£, d; u*) 

i- 

H d£ 

£=x n (0)_ 


dh (x n (d),s-,T) = ^[J°(x n (0),s-T)-J°(x n (0),s-d) 

= e~ rs [/(x n (0); T ) - /(x n (0); ^AT^O); T) 

i e- rs (MT)A(x n (0)-T)A~\x n (0)-i9)-M^)/K- 


-i 


Since all these mappings are continuous, for T = r n ,s = 0, we obtain 


d} j (—J t )(x n (t), 0; r n ) = {if n (t)/X n }; (8) 

9i J (—J t )( x n{t)-i 0; T n ) = {(f>n(t)/ A n , r J l (x n (t), 0; r n ))}. (9) 

Let us also note that A n —> X* = 0 exactly when —^(O) = X n I(x n (0);r n ) —> —-0*(0), i.e., 
when ||/(x n (0);r n )|| ->• oo. 


3 The main result 

Theorem 3 Let the process (x*,u*) be uniformly overtaking optimal for problem (I To II - (I Tc p . 
Assume condition m to hold. Take an arbitrary unboundedly increasing sequence of times 
T n - 

Then, for (x*,u*) there exists a nontrivial t- limiting solution (ip*,X*) of system (1 36 1) - 
(1 3c I) for X* G {0,1} such that 


H(x*(t),u*(t),if*(t),X*,t) = if*(t)f(x*(t),u*(t)) - X*e rt f 0 (x*(t),u*(t)), 





















for almost all t, coincides with a continuous function H* : T —» M , and, for all T e T , the 
function H* satisfies 


(vanishing of Hamiltonian) 
(stationarity condition) 


lim H*[t] 

t—>oo 


— H*[T] 


(transversality condition at zero) -0*(O) 

(limiting condition for x) if*(T) 

(limiting condition for (x,t)) (if*(T), — H*[T ]) 


= 0 ; ( 10 a) 

= A *r lim J T (x*(T)-,r n ) ( 10 b) 

n—too 

= A*r(J** — J°(x*(0), 0; T)) 

/ oo 

e~ rt f 0 (x*(t),u*(t)) dt ; 

e A*<9 L /(:r*(0)) + N%(x*(0)); (10c) 

G d£*(—J t )(x*(T); oo t ); (lOd) 

e df (-J t )(x*(T),0;oo t ). ( 10 e) 


Moreover, if X* = 0, then 0) 7 ^ 0 holds, the sequence l(x n (0), r n ) is unbounded, and for 
almost all T > 0 


iT[T] = 0 VT > 0; 

if*(T) f (x*(T), u*(T)) = 0 V a.a. T > 0. 


(100 

(log) 


Note that if / 0 is bounded and r > 0, then (CO holds. Such assumption is used, for 
example, in [31] 

Let us also make several simple observations. 


Corollary 1 Under assumptions of the theorem, let r 


we also have ( 10 / ) and 


0 ; then, in addition to ( 1 10 a I) - (I TUe p , 


if*(T)f(x*(T),u*(T)) = X*f 0 (x*(T),u*(T)) Wa.a.T> 0. (lOh) 


Another one of them is about the converse of Hartwicks rule in resource economics (see 


Corollary 2 Under assumptions of the theorem, let f 0 (x*(t),u*(t)) = C hold true for a certain 
constant C for almost all t > 0. Then, there exists a nontrivial t- limiting solution (if*, A*) 
of system (1361) - (1 3c p for which, in addition to (1 10a I) - (I lOe p , ( | 10 g p also holds. 

Proof. Indeed, replace f 0 (x,u) with the function f c (x,u) = f 0 (x,u) — C. The optimal process 
remains optimal, condition (fTj) holds, and the solution of the PMP does not change. Apply 
the proved theorem. Then, ( | 10 g | ) holds for almost all T > 0 by virtue of 

H* C [T\ = if*(T)f(x*(T),u*(T)) - X*e~ rT f c (x*(T),u*(T)) = if*(T)f(x*(T),u*(T)), 


-H* C [T} V A *r 


e r f c (x*(t),u*(t)) dt = 0 . 


□ 
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Corollary 3 In some neighborhood 17 of the point 5* , let the value function V T (b) of the 
problem 

r 

Minimize / fo(x,u)dt 

Jo 

subject to x = f(x,u), u EU 

x( 0 ) = b. 

be such that, for a Lipshitz function V°° defined in that neighborhood and a number H°° G R , 
we have 


lim [ V T (b ) + H°°T] = V°°{b ) Vb G ft, (11) 

and this limit is uniform for b G 57. 

In addition, let the control u* G U satisfy 

Hm [J°(K, u*-T ) + H°°T] = V°°(K). (12) 

Then, for (x*,u*), there exists a r-limiting solution (if*,\*) of the PMP relations such 
that X* = 1 and 


r(u)£di(-v){h) 

r(T)f(x*(T),u*(T)) = f 0 (x*(T),u*(T)) + H° 
Proof. For every T > 0, b G 17 , consider the problem 


V a.a. T > 0. 


(13) 

(14) 


Minimize — V T (b) + / [f 0 (x,u) + H°°]dt 

Jo 

subject to x — f(x,u), u E U, 

x(0) = b. 

Without loss of generality we can assume that 17 is closed. It is easy to see that the function of 
optimal value for this problem equals H°°T . Note that conditions (1 12 1) and (1111) now imply 
condition m, as well as the fact that u* is uniformly overtaking optimal in the problem 
(TToD - ((TcJ (with r = 0 , G = 17) 

Then, the result of the theorem holds for it; in particular, there exists ^*(0) G 
A *df(—V)(b*) + N2(K). Since N^(b^) = {0}, from A* = 0 one would imply ^*(0) = 0, 
which contradicts the nontriviality of the r-limiting solution. Then, A* = 1 and (1 13 1) . Writ¬ 
ing out ( 1 10 /?. j) for this problem, we obtain (1 14 1) . □ 

Note that condition (1 13 1) is nothing else but the economical interpretation of a co-state 
arc as a shadow price. It is proved under varying assumptions on the system, for example, in 

mummm- 
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4 Auxiliary lemmas. 

Let E, T be a finite-dimensional Euclidean spaces. Consider a map a : E x T xTh> E 1 . 

As for the class of admissible controls, we consider any nonempty subset of measurable 
functions a(-) bounded for any time compact such that a(t) G T holds for a.a. t G T. 
Denote the set of admissible controls by A. 

For each admissible control a G A , consider the differential equation: 

y = a(y(t), a(t),t ), Vt > 0. (15) 

Assume that, for each admissible control a , the map (y, t ) t-x a(y, a(t),t ) is a Caratheodory 
map; on each bounded subset, a map (y,a,t) ^ a(y,a,t ) is integrally bounded and locally 
Lipshitz continuous on x; moreover, each its local solution of (1151) can be extended onto 
the whole T [28|. For every admissible control a, let us denote the family of all solutions 
y G C(T,E) of system (1 15 p by Y[a]. 

Consider any admissible control a* G A and a compact set S . Let us fix a*. S . 

For every point (?/*, $) G Ex T , there exists a unique solution y G C(T, E) of the equation 

y = a{y{t),a*{t),t), y{$) = y*. (16) 

Let us denote its initial position y(0) by x(y*,i?). 

In [IT] for such system a with the designated control a* and the compact set S, the map 

w: TxTxTgT 

was constructed. It has the following properties: 

• the mapping T9 (ig w(a'(t), a"(t),t ) is Borel measurable for each a', a" G A ; 

• the mapping T x T 3 («',«") e-x w(a',a",t) is lower-semicontinuous for a.e. t G T; 

• for any (a', a", t) G T x T x T, w{a', a", t) — 0 iff a' — a" ; 

• for any (a’,a",t) GlxTxT, w(a',a",t) = w{a",a!,t ) > 1 if a' ^ a" ; 

• the following lemmas hold (see Em Lemmae A.1-A.3]): 

Lemma 1 For every T > 0, the mapping 

A 9 (a 1 , a") i—x p(a',a",T) = f w(a'(t),a''(t),t)dt 

Jo 

defines a metric on 

At = {aGd| a(t) = a*(t ) Vf > T}; 


11 






under this metric, the space At becomes a complete metric space, and the convergence in this 
metric is not weaker than the convergence in measure. 

In particular, if for some unbounded increasing sequence of times r n , for some sequence of 
a n G A Tn , the sequence of p(a*, a n , r n ) tends to zero, then the sequence of a n converges in 
the measure to a* on the whole T. 

Lemma 2 For arbitrary a G A, T > 0, every solution y G Y[a],?/(0) G S of equation (1 15 ]) 
satisfies 

11 x(y (t), t) - 2/(0) 11 < p(a*,a,t) V£g[0,T]. (17) 

if p(a*,a,T) < dist(y(0),bd§>). 

Lemma 3 For a sequence of a n G A and a sequence of y n G Y[a n ], let 

p(a*, a n , T ) —>■ 0, y n (0 ) —* £ as n —* oo 

for some T > 0, £ G int S . 

Then, the solutions y n converge to the solution of (1 15 |) generated by a* from the point £ , 
and this convergence is uniform in [0, T\ . 

Hereinafter set T = U x [1/2, oo), A — U x B(T, [1/2, oo)), a* = (u*, 1). 

We will require the following property, which was essentially proved by Michel in [ 191 
Lemma]: 

Lemma 4 Consider Borel-measurable mappings u G U,v G B( T, [1/2, oo)) and the solutions 
of the system generated by them 

V = v(t) f(y(t),u(t)), x(0) = &; 

i = v(t), ^(0) = 0. 

Then, there exists a control u' G II and the trajectory x' = x(b,u •) generated by it such that 
x'{z(t )) = x(t) for all t G [0, r n \ and 

rr n 

/ v(t)e~ rz{t) f 0 (x(t),u(t))dt= e~ rt fo(x'(t),u'(t))dt. (18) 

Jo Jo 

Proof. Note that every such map z : T —> T is continuous, strictly increasing, and reversible; 
denote the inverse map of 0 by (. It would then suffice to set u'(s) = u(£(s)),x'(s) = x(((s)) 
for all s < z(r n ). As proved in [TD, Lemm], in these circumstances, y' = x{b*,u'] •) and (1 18 1) . 
□ 

For an unbounded sequence of positive numbers r n , dehne the scalar function h n by the 
following rule: 

Ms) = e ~ rs (J** - J°(K, 0; r n )) Vs G M> 0 . 
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Note that 0 now implies 


lim sup h n (s) = 0 (19) 

n— *°° s£[—1,1] 


Lemma 5 Suppose that u* is a uniformly overtaking optimal control of original problem 
(I la p - (I lcl) ; i.e. in problem 

poo 

Minimize 1(b) + / e~ rt fo(x, u) dt 

Jo 

subject to x = f(x,u), u G U, 

x(0) G G. 


Assume the number J** to be validly defined by (T2~j) . Take an arbitrary unboundedly increasing 
sequence of times r n . 

Assume that, for some unbounded sequence of positive numbers r n , a sequence of functions 
h n G C (M, M) satisfies (1 19 II . 

Then, the sequence of optimal values of the problems 


hn(z(T n ) - r n ) + l(x( 0 )) + [ v(t)e rz{t) fo(x(t),u(t))dt ( 20 a) 

Jo 

subject to x = v(t) f (x(t),u(t)), z — v(t); ( 20 b) 

t > 0, u(t) G U, |u(i) — 1| < e _t ; (20c) 

x(0) G C, z(0) = 0 (20d) 


converges to l(b*) + J**. 


Proof. Note that the control (u*, 1) is admissible for problem (1 20a I) - (1 20 d I) . and, by the 
definition of J** and (1 19 p , it provides the value of payoff that is arbitrary close to /(&*) + J** 
(for large n ). 

By condition, there exists a sequence of positive u n that converges to zero such that h n (t) < 
io n if |t| < 1 for every n G N. 

Assume the implication of the lemma to be false. Then, there exist a positive number £, 
a sequence of initial conditions b n G C, and a sequence of controls (u n ,v n ) with (1 22c I) such 
that, for any natural n, the trajectory (x n ,z n ) generated by the control (u n ,v n ) from the 
position (fe n , 0 ) satisfies 

l(b n ) + h n (z n (r n ) - r n ) + f v n (t)e~ rZn{t) fo(x n {t),u n (t)) dt < Z(6*) + J** - 4s. 

Jo 

Since we also have \z n (r n ) — l\ < e _t , we now know that \z n (T n )— r n \ < 1, i.e. \h n (z(r n )— r n )| < 
ui n . Now, for all n starting with a certain one, 

l(bn) + r v n (t)e- rz ^fo(x n (t),u n (t)) dt < l(b*) + J** - 3e. 

Jo 


13 














Thanks to Lemma HI for sufficiently large n, there exists the control u' n G II and the 
trajectory x' n = x(b n ,u' n \ •) generated by it such that (1 18 1 ) holds, whence 

rzn(r„) 

l(b n )+ / e- rt f 0 (x' n (t),u' n (t))dt<l(h) + r*-2£. 

Jo 

Now, z n (r n ) —* oo and (T2~j) imply that, for sufficiently large n G N, 

r^n (j~n ) r^n ('Tn ) 

^«( 0 )) + / e ~ rt fo(x' n (t),u' n (t)) dt < l(K) + / e~ rt f 0 (x*{t),u*(t)) dt - e. 

Jo Jo 

However, it contradicts the fact that (x*,u*) is a uniformly overtaking optimal process for 
problem (I lap - (I Tc ]) . □ 

This allows us to proceed to the actual proof of the main result. 


5 Proof of Theorem EL 


5.1 Choosing the metric p. 

Consider the following system: 

x = vf(x,u)\ (21a) 

z = v; (21b) 

i> = -v ^-(x,u) + \ve~ rz ^-(x 1 u); (21c) 

<j) = —\rve~ rz fo(x, u); (21d) 

A = 0. (21e) 


Remember that T = U x [1/2, oo). Let fi be a ball in X centered at 6* with the radius 
1/2. Set £ = XxRxIxlxl, y* = (6*, 0, 0, 0, 0) G E. Let S be a ball in E centered at 
?/* with the radius 2. 

Let the mapping a : E xT xT —> E be the right-hand side of system (1 2 In I) - (1 21e I) . This 
system satisfies all the requirements we demand from a system (1 15 II . Consider mappings w, p 
for such system a with designated control a* = (u*, 1 ) and the compact set S. 

Remember that A — 'll x R([l/2, oo)), and, for each n G N, 

A Tn = {a = (u, v) e A | u(t) = u*(t),v(t) = 1 Vt > r n }. 

By Lemma [H A Tn is metrizable by («', a") t-x p(a!,a",r n ). 
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5.2 Constructing the auxiliary optimal solution sequence. 


By Lemma [51 there exists a sequence of positive numbers y n converging to zero such that, 
for any natural n , the optimal value for (I 20 a ]) - (1 20 dl) is bounded from below by the value 
/( 6 *) J ** — 7 ^. Then, it is also a bound from below for the value of the following auxiliary 
minimum problem: 

l(x( 0 )) + / v(t)e^ rz ^f 0 (y(t),u(t))dt + h n (z(r n ) - r n ) 


Jo 

+7 n p(oi*,oi,T n ) + 7 n ||x(0) - 6*11 (22a) 

subject to x — v(t) f(x(t), u(t)), z = v(t); ( 22 b) 

t > 0, a(t) = (u(t),v(t)), u(t) G U, |r>(t) — 1| < e _t ; (22c) 

i(o)eenfl, ^(o) = o. (22d) 


Consider the set of all admissible controls a = (u, v ) in this problem. This set contains 
a* = (u*, 1), and is a subspace of the complete metric space A Tn . 

By the Ekeland principle P Theorem 5.3.1], [11 , Theorem 2.1.3], for problem (1 22a ]) - 
(1 22 d I) , there exists an optimal pair (6 n , a n ) in the complete metric subspace of (Cflhl) x A Tn ; 
denote by (x n ,z n ) its solution of (1 22 b I) . fl 22d [) . Moreover (see [6j Theorem 5.3.1.(i)].[TTl 
Theorem 2.1.3,(ii)]), 

/(&*) + J°(6*,0,u*;r n ) + h n (0) > [ v n (t)e~ rZn ^ f 0 (x n (t), z n (t)) dt 

Jo 

T/(x n (0)) T h n (^z(^T n ^j T n ) 

+7 n p(cc*, a n , Tn) +7npn(0) - 6*11, (23a) 

||x n (0) - 6*|| +p(a*,a n ,r n ) < y n ->• 0 as n —>■ oo. (23b) 


From (1 22 b I) and (1 22c li one can readily obtain | z n (r n ) — r n \ < 1; now, (1 19 1) implies 

dh n 


ds 


Let us show that 


{z n (r n ) - r n ) = -rh n (z n (r n ) - r n ) —>■ 0 . 


v n (t)e rZn{ -J f^x n {t),z n {t))dt ->■ J**. 


(23c) 


(23d) 


Indeed, to prove that the upper limit does not exceed J**, it is sufficient to pass to the limit 
in (I 23a ]) using (1 19 ]) , fl 236 ]) . On the other hand, as it was noted before, the integral can be 
estimated from below by the value Z( 6 *) — l(b n ) + J** — 7 ^ by virtue of Lemma 0 However, 
the limit of this expression is also equal to J**. Thus, (1 23d I) is proved. 

Note that, by (1 23 b I) and Lemma [U a n = (u n ,v n ) converges in measure to a* = (u*, 1) on 
the whole T. Passing to the subsequence if necessary, we can say that (u n ,v n ) converges to 
(u*, 1) a.e. on T. 
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5.3 Pontryagin Maximum Principle for auxiliary problem. 


Since a n = (u n ,v n ) provides a minimum of problem (1 22a I) - (1 22d I) , it can, if need be, yield 
the Pontryagin Maximum Principle [12, Theorem 5.1.1]. Without loss of generality, we may 
(1 23b I) assume that x n (ti) G int Q for all n G N. Then, iVf(5; re (0)) = ^ ne (T n (0)). 

Let the function H n :XxfxY xIxlxTxTi-fK be given by 

H n (x,z,u,v,i/i,<j>, X,t) = ipvf(x,u) + (f)V - Xve~ rz f 0 (x,u) - Ay n w(a*(t), (u,v),t). 

Then, by the Maximum Principle, there exist A n G (0,1], Ln G C(T, X), (j) n G C7(T, M) with 


A + |</>n(0)| + ||'0n(O)|| — 1 

such that, for some Q G X(||£|| < 1), the transversality conditions 

Ln{ 0) G A„9],Z(x n (0)) + Xn'YnC + JV£(z n (0)), 
-4>n(r n ) = xj^{z n (r n ) - r n ), 


lY, ( 7n ) 0 


(24a) 

(24b) 

(24c) 

(24d) 


hold, and 


- V’nW = 


OIL 

dx 


(x n (t),U n (t),V n (t),Ln(t), 0„(t), X n , t) 




= v n (t)^(x n (t),u n (t))-XnV n (t)e rZn{t) ^(x n (t),u n (t))-, 
0H n - - 


dz 


' (-^n D i 'Un(L) i (^)) Ln (Z) j 0n (Z) 5 A n , f ) 


= A n rv n (t)e rZn{t) fo(x n (t),u n (t)); 
sup H n (x n (t),u', v', Ln(t), <i>n(t), A n , t) 

, u / E£/,|t; / —l|<e — * 

H n 5 ^n(^)? ^n(^)5 1 0 n(^)5 

also hold for a.e. t G [0, r n ]. 


(24e) 


(24f) 


(24g) 


5.4 Pontryagin Maximum Principle for overtaking optimal process 


Set y n = (y n , z n , Lm <t>m X n ) for each n G N; note that this is a solution of (I 21a p - (I 21e p . 

By (1 24a I) , passing, if need be, to a subsequence, we can consider the subsequence of 
X n G (0,1] to tend to some A* G [0,1] and a subsequence of (^ n (0), 0 n (O)) to converge to a 
certain (^o,0o) £ X x M. as well. We now have, for sufficiently large n, 


j~247n 

||?/n(0) - y*\\ < X* + HV’oll + I0SI + lbn(0) - 6*11 < l + 7 n< 2 . 


(25) 
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Thus, y n ( 0) ->■ (6*, 0,-00, (ft* 0 , X*) G mtS. 

In addition, for each T > 0, we have p(cn*,a,T) < p(a*,a,r n ) for all u G if if T < r n ; 

now, from (1 2361) . we have p(a*,a n ,T) —> 0. Therefore, by Lemma EJ in every compact 

interval, the subsequence of uniformly converges to a solution y* of system (1 21a II - 
(1 21e I) generated by the control (u*, 1), i.e. , the subsequence of (x n , ift n , (ft n , X n ) converges 
to the solution of (1 4a I) - (1 4c p . Moreover, y*(0) = (6*, 0, ift^, A*). Then, y* has the 

form y*(-) = (x*(-), •, ift*(-), </>*(•), A*), where functions ift*, (ft* are solutions of (1 46 [) and of 

(ft* = —X*re~ rt f 0 (x*(t),u*(t)) with initial conditions ift*(0) = iftft,, </>*(0) = <ft* 0 . 

Remember that (u n ,v n ) converges a.a. to (u*, 1) . Then, w((u*(t), 1), (u n (t), v n (t)), t) —> 
w((u*(t), 1), (u*(t), l),i) = 0 for a.e. t G T. Now, passing to the limit in ( | 24 g [ ), we have, for 
a.e. t G T , 


sup 

uGU,\v— 11<e * 


i>*(t)vf(x*(t),u,t) +v<f>*(t) 


X*ve rt f 0 (x*(t),u,t) 


*(t)f{x*(t),u*(t),t ) + 4>*(t) — X*e rt f 0 (x*(t),u*(t),t). 


(26) 


Setting v = 1, we obtain (1 3c p for (x*,ift*,X*) for almost every t > 0. Thus, the limit 
(x*, ift*, A*) satisfies system (1 3a ]) - (1 3d I) for u = u* , i.e., system (1 4a p - (1 4c p . 


5.5 Backtracking 

Since (1236 1) and (1 25 1) imply p(a*,a n ,r n ) < q n < 1/2 < dist(y n (0),bd§), and y n —3 > y*, 
q n —> 0 as n —> oo , we know that Lemma [2] guarantees 

x(yn(T n ),T n )->y*( 0). (27) 

From the position y n {r n ), launch in reverse time a solution y n of system (I 21 a I) - (1 21e I) 
with the help of the control (u*, 1). Then, y n ( 0) = >c(y n (r n ), r n ) (see (1 161) 1. Note that 
y n = (x n ,z n ,if> n ,<j> n , X n ) satisfies (T4a~D - d~4c~|) , and ift n (T n ) = ift n (r n ) = 0, (ft n (r n ) = (ft n (r n ) = 
— ^(z n (r n ) — r n ). By the theorem on continuous dependence of the solution of a differential 
equation, (1 27 1) implies that the solution y*(-) = (x*(■),•, X* , ift*(■), (ft*(■)) is the limit (in the 
compact-open topology) of y n . 

Note that the mappings 6 (->■ d\l(b ), b (->■ TV/(6) are upper semicontinuous; passing to the 
limit in (I 246 1) , we see that -0„(O) —» -0*(O) and 5v(0) —> x*(0) = 6* imply (1 10c p . 

Since the supremum in (1 261) contains a function that is linear in v and that attains 
its maximum in v at the interior point v — l, we have ift* (t) f (x* (t), u* (t ), f) +</>*(£) — 
X*e~ rt f 0 (x*(t),u*(t),t) = 0 for almost every t G T, i.e., <ft*{t) = —H (x* (t), u* (t), ift* (t), X*) for 
almost all t G T. 
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Now, — H(x*(t), u*(t), A*) coincides with the limit of (f> n . Thanks to (121c/|) and 

(1 24c p , every 0 n also satisfies 

0„(i) = ~Kre~ rXn ®f 0 (x n (t),u*(t)), ~^ n (r n ) = xj^(z n {r n ) - r n ). 

Then, for all T > 0, the Hamiltonian H*[T] coincides with 


lim 

n—too . 


lim A„r 

u->oo 


I23H 

EM 


/ Tn ~ dh 

X n re~ rZn( - t) f 0 (x n (t),u*(t))dt + X n —jj-(z n {T n ) - r n ) 

e - rZn{t) fo(x n (t), U *(t))d t + [ e~ rZn{t) f Q (x n (t),u*(t))dt 

Jo 

—A*r J** + lim A n r f e~ rZn ^fo(x n (t),u*(t)) dt. 

n^oo J 0 

Passing to the limit, in view of the theorem on continuous dependence of the solution of a 
differential equation on its initial conditions, we obtain 

rT 


H*[T] = -A*rJ** + A*r / e~ rt fo{x*(t),u*(t)) dt 

Jo 

= A *r[-J» + J°(h,u*-,T)]. 


(28) 


Thus. (| 106 1) is proved. Expression TO now implies (1 10a I) . 

Note that although the constructed sequences converge to (0 q, 0*, A*) such that | IV'o 11 + 
|0o|+A* = 1, we have ||V , oll+A* > 0 . Indeed, we would otherwise have 0* = 0, A* = 0, whence 
0*(O) = 1 and H(y*(t), u*(t), 0*(t), A*) = 0, i.e. H* = 0, which contradicts H* = —0*. Thus, 
|\ ipQ 11 + A* > 0. Note that, since ||0 n (O)|| ^ A n ||/(a: n (0), r n )||, A* = 0 exactly when the 
sequence from ||/(a; n (0), r n )|| is unboundedly increasing. 

In case of A* > 0, A* yJ 1 note that relations (1361) , (1 3c I) , (1 2461) - ( 24/ ) are preserved under 
multiplication of (0*,0*,A*) along with the subsequences (0„, 0 n , A n ) by a positive number. 
Hence, by multiplying the triple (0*,0*,A*) along with the subsequences (if) n ,<f> n , A n ) by the 
number d_, we provide A* = 1. Thus we can safely assume A* G {0,1}. 

Expressions (T8~l) , fHJT) imply that, for all T > 0 for each n G N, 

0 n (T) G A n d L (-J T )(x n (T)-,u*,T n )-, 

(jp n (T)i X n r J T (x n (T), 0; r n )) G X n d\{-J T ){x n (T), 0; r„). 


Passing to the limit as n —* oo , by virtue of X n —> X*, x n —* x*, —$■ ip *, and (1 106 1) , we 

have (I lOrf j) , (I lOe p . 


In case A* = 0, (1 28 p implies that H* = 0, whence we obtain Q 10/ D ; setting A* = 0 in 

□ 


d 10/ , we obtain (| 10 g |) . 
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